Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent.

Time series classification is crucial for numerous scientific and engineering applications. In this article, we present a numerically efficient, practically competitive, and theoretically rigorous classification method for distinguishing between two classes of locally stationary time series based on their time-domain, second-order characteristics. Our approach builds on the autoregressive approximation for locally stationary time series, combined with an ensemble aggregation and a distance-based threshold for classification. It imposes no requirement on the training sample size, and is shown to achieve zero misclassification error rate asymptotically when the underlying time series differ only mildly in their second-order characteristics. The new method is demonstrated to outperform a variety of state-of-the-art solutions, including wavelet-based, tree-based, convolution-based methods, as well as modern deep learning methods, through intensive numerical simulations and a real EEG data analysis for epilepsy classification.

Summary: The paper consider the setting of streaming PCA for time series data which contains two challenging ingredients: data stream dependence and a non-convex optimization manifold. The authors address this setting via downsampled version of Oja's algorithm. By closely inspecting the optimization manifold and using tools from the theory of stochastic differential equations, the authors provide a rather detailed analysis of the convergence behavior, along with confirming experiments on synthetic and real data. Evaluation: Streaming PCA is a fundamental setting in a topic which becomes increasingly important for the ML community, namely, time series analysis. Both data dependence and non-convex optimization are still at their anecdotal preliminary stage, and the algorithm and the analysis provided in the paper form an interesting contribution in this respect.

The mean and variance of the series are constant over time. The series has a constant trend over time. The auto-covariance function of the series is dependent on time. The series has a periodic pattern over time. A moving average uses past errors, while an autoregressive model uses past values of the dependent variable. A moving average uses only one past value, while an autoregressive model uses multiple past values.

In this paper, we use the blocking technique to demonstrate the effectiveness of GANs for estimating the distribution of stationary time series. Theoretically, we derive a non-asymptotic error bound for the Deep Neural Network (DNN)-based GANs estimator for the stationary distribution of the time series. Based on our theoretical analysis, we propose an algorithm for estimating the change point in time series distribution. The two main results are verified by two Monte Carlo experiments respectively, one is to estimate the joint stationary distribution of 5-tuple samples of a 20 dimensional AR(3) model, the other is about estimating the change point at the combination of two different stationary time series. A real world empirical application to the human activity recognition dataset highlights the potential of the proposed methods. Estimation of distribution plays an important role in data analysis. Many traditional methods on distributional estimation are based on nonparametric kernel methods, and suffer from the curse of dimensionality.

The first three approaches exploit differencing to make stationary the time series. Firstly, I import the dataset related to tourists arrivals to Italy from 1990 to 2019 and convert it into a time series. Data are extracted from the European Statistics: Annual Data on Tourism Industries. I use the matplotlib library. Usually, when performing time series analysis, a time series is not split into training and test set, because all the time series is needed to get a good forecast. However, in this tutorial, I split the time series into two parts -- training and test -- in order to test the performance of the tested models.

In this story we will shed some light on something quite unsettling regarding time series conintegration, that is model overfitting. Usually, we associate overfitting with other kinds of models, but not with conintegration, after all, if we find a stationary linear combination isn't that enough? It turns out that it is not. When we increase the number of time series used to find cointegration relationships, in most cases, we see that the resulting time series keeps getting lower Dickey-Fuller test values, so it becomes more and more likely that such a time series is stationary. For instance, if we use several Brownian motions (AR(1) unit root processes) it is easy to get Dickey-Fuller results of about -10.

In the previous blog, we discussed a predictive model for house prices using Machine Learning algorithms. In this blog, we are going to discuss the time series forecasting on Zillow economics data using a statistical modeling approach. The project was implemented in September 2019 and forecasting of house prices was done for the next year that is 2020. The code could be reused by changing the span of forecasting that is year for forecasting or duration of forecasting. The results discussed in this blog are for the year 2020.

Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent. Specifically, our goal is to estimate the principle component of time series data with respect to the covariance matrix of the stationary distribution. Computationally, we propose a variant of Oja's algorithm combined with downsampling to control the bias of the stochastic gradient caused by the data dependency. Theoretically, we quantify the uncertainty of our proposed stochastic algorithm based on diffusion approximations.

Numerous control and learning problems face the situation where sequences of high-dimensional highly dependent data are available but no or little feedback is provided to the learner, which makes any inference rather challenging. To address this challenge, we formulate the following problem. Given a series of observations $X_0,\dots,X_n$ coming from a large (high-dimensional) space $\mathcal X$, find a representation function $f$ mapping $\mathcal X$ to a finite space $\mathcal Y$ such that the series $f(X_0),\dots,f(X_n)$ preserves as much information as possible about the original time-series dependence in $X_0,\dots,X_n$. We show that, for stationary time series, the function $f$ can be selected as the one maximizing a certain information criterion that we call time-series information. Some properties of this functions are investigated, including its uniqueness and consistency of its empirical estimates. Implications for the problem of optimal control are presented.